ATOM Documentation

ā† Back to App

CI/CD Coverage Enforcement

**Phase:** 35 Plan 06

**Status:** āœ… Configured

**Last Updated:** 2026-02-15

Overview

The platform has automated coverage enforcement via GitHub Actions that runs on every pull request and push to main branch. This ensures code coverage never regresses below established thresholds.

Coverage Thresholds

Frontend (Vitest)

  • **Overall:** 80% lines, functions, statements; 70% branches
  • **Brain Systems (src/lib/ai):** 90% lines, functions, statements; 80% branches
  • **Integrations (src/lib/integrations):** 85% lines, functions, statements; 75% branches

Backend (pytest)

  • **Overall:** 80% coverage
  • **Critical Files:** 90% coverage (episode service, graduation exam, governance)

GitHub Actions Workflows

1. Coverage Enforcement Workflow

**File:** .github/workflows/coverage.yml

**Triggers:**

  • Pull requests to main branch
  • Pushes to main branch

**Jobs:**

Frontend Coverage

  • Runs npm run test:coverage
  • Enforces 80% threshold (fails CI if below)
  • Checks brain system files for 90% threshold
  • Generates JSON, HTML, and lcov reports

Backend Coverage

  • Runs pytest with --cov=core --cov=api
  • Enforces 80% threshold (fails CI if below)
  • Generates JSON and terminal reports

Coverage Report

  • Combines frontend and backend coverage
  • Comments on PR with coverage summary
  • Shows coverage vs target comparison
  • Indicates pass/fail status

Status Badge

  • Updates coverage badge on push to main
  • Stores badge artifact for 90 days
  • Color-coded: green (ā‰„80%), yellow (ā‰„60%), red (<60%)

2. Test Coverage Workflow

**File:** .github/workflows/test-coverage.yml

**Features:**

  • Uploads coverage to Codecov (optional)
  • Generates combined coverage reports
  • Uploads HTML reports as artifacts
  • Shows coverage targets in GitHub summary

PR Coverage Comments

Every PR automatically receives a coverage comment:

## šŸ“Š Coverage Report

### Frontend

| Metric | Coverage | Target |
|--------|----------|--------|
| Lines | 82.5% | 80% |
| Functions | 78.3% | 75% |
| Branches | 72.1% | 70% |
| Statements | 81.2% | 80% |

**Status:** āœ… Meets 80% threshold

### Backend

**Overall Coverage:** 85.2%

**Status:** āœ… Meets 80% threshold

---

*This comment is automatically generated by the coverage enforcement workflow.*

Coverage Badges

Badges displayed in README.md:

[![Frontend Coverage](https://img.shields.io/badge/coverage-80%25%20target-green)](...)
[![Backend Coverage](https://img.shields.io/badge/backend-80%25%20target-blue)](...)
[![Test Status](https://img.shields.io/badge/tests-98.3%25%20passing-brightgreen)](...)

CI/CD Enforcement Behavior

On Pull Request

  1. **Coverage Check Runs:**
  • Frontend coverage job starts
  • Backend coverage job starts
  • Both run in parallel
  1. **Threshold Enforcement:**
  • If coverage < 80%: āŒ CI fails, PR cannot merge
  • If coverage ā‰„ 80%: āœ… CI passes, PR can merge
  1. **PR Comment:**
  • Automatic coverage summary posted
  • Shows pass/fail for each metric
  • Includes coverage vs target comparison
  1. **Status Checks:**
  • "Frontend Coverage (80%)" must pass
  • "Backend Coverage (80%)" must pass
  • Both required for merge

On Push to Main

  1. **Coverage Check Runs:**
  • Same as PR check
  1. **Badge Update:**
  • Coverage badge updated with latest percentage
  • Badge color changes based on coverage level
  1. **Report Storage:**
  • Coverage reports stored as artifacts (30-90 days)
  • HTML reports available for download

Running Coverage Locally

Frontend Coverage

# Run coverage with report
npm run test:coverage

# Open HTML report
npm run coverage:report

# Check specific file coverage
npx vitest run --coverage --reporter=verbose src/lib/ai/cognitive-architecture.ts

Backend Coverage

# Run coverage with report
cd backend-saas
pytest --cov=core --cov=api --cov-report=html --cov-report=term-missing

# Open HTML report
open htmlcov/index.html

# Check specific module coverage
pytest --cov=core.episode_service --cov-report=term-missing

Troubleshooting

CI Fails with "Coverage below threshold"

**Cause:** New code reduced coverage below 80% (or tier-specific threshold)

**Solution:**

  1. Check the PR comment for specific metrics that failed
  2. Add tests for uncovered code paths
  3. If decrease is intentional (e.g., removing dead code):
  • Update tests to cover new code
  • Document why coverage decreased
  • Get team approval before merging

Coverage Report Not Generated

**Cause:** Test failures prevent coverage generation

**Solution:**

  1. Fix failing tests first
  2. Coverage report generates only when all tests pass
  3. Use --no-coverage flag to run tests without coverage:
npm run test:coverage -- --no-coverage

Brain System Threshold Failure

**Cause:** AI system files below 90% coverage

**Solution:**

  1. Check which brain system file failed the threshold
  2. Review HTML coverage report for uncovered lines
  3. Add tests for uncovered functions/branches
  4. Core brain systems (cognitive, learning, world model) need 95%

Backend Coverage Fails

**Cause:** Backend coverage below 80%

**Solution:**

  1. Check backend coverage report: backend-saas/coverage.json
  2. Identify modules with low coverage
  3. Add unit tests for critical backend files
  4. Focus on: episode service, graduation exam, governance, business intelligence

Coverage Reports

Artifacts (GitHub Actions)

After each CI run, coverage reports are stored as artifacts:

  • **frontend-coverage:** JSON and HTML reports
  • **backend-coverage:** JSON and HTML reports
  • **coverage-summary:** Combined markdown summary
  • **coverage-badge:** Badge JSON for README

Local Reports

  • **Frontend:** coverage/index.html
  • **Backend:** backend-saas/htmlcov/index.html
  • **Frontend JSON:** coverage/coverage-summary.json
  • **Backend JSON:** backend-saas/coverage.json

Coverage Trends

To track coverage over time:

  1. Check badge in README for current coverage
  2. Review PR comments for coverage changes
  3. Download artifacts from GitHub Actions for historical data
  4. Use Codecov (if configured) for trend visualization

Best Practices

  1. **Write Tests First:** TDD approach ensures coverage from the start
  2. **Check Coverage Locally:** Run npm run test:coverage before pushing
  3. **Review PR Comments:** Check coverage summary before merging
  4. **Fix Coverage Drops:** Don't merge PRs that reduce coverage
  5. **Focus on Critical Code:** Brain systems and governance need highest coverage
  6. **Document Exceptions:** If coverage must decrease, document why and how to fix

Configuration Files

  • **Frontend:** vitest.config.ts (thresholds, report directories, per-file thresholds)
  • **Backend:** pytest.ini and pyproject.toml (coverage settings, thresholds)
  • **CI/CD:** .github/workflows/coverage.yml (enforcement, PR comments, badges)
  • **Codecov:** .github/workflows/test-coverage.yml (optional upload to Codecov)

Related Documentation

Success Criteria

āœ… **All criteria met:**

  • [x] GitHub Action workflow created for coverage checks
  • [x] PR checks fail if coverage < 80%
  • [x] Coverage reports commented on PRs
  • [x] Coverage badges added to README
  • [x] CI/CD enforcement tested and working
  • [x] Documentation created

Next Steps

  1. **Monitor Coverage:** Watch PR comments for coverage trends
  2. **Fix Failures:** Address coverage drops immediately
  3. **Incremental Improvement:** Gradually increase thresholds (80% ā†’ 85% ā†’ 90%)
  4. **Report Generation:** Run full coverage report after each phase
  5. **Badge Updates:** Badges auto-update on push to main

---

**Phase 35 Status:** Coverage enforcement configured and active. All PRs must meet 80% threshold to merge.